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(3) Method for synchronizing Interconnected digital equipment 



© Optimized synchronization planning and dock distribution for a network of Interconnected digital 
equipment is achieved by designating a network node at the highest stratum level as the master clock 
node, forming a group of all unassigned nodes connected to the assigned node or nodes, selecting 
subgroup of ail nodes from the group wherein the subgroup Includes all nodes having the highest 
stratum level of the group, limiting the subgroup to the nodes which have a desired characteristic when 
such nodes are Included In the subgroup, determining the synchronization performance of each node 
in the subgroup according to a predetermined criterion, assigning one node from the subgroup as a 
clock timing receiver wherein the one node exhibits the best performance for nodes in the subgroup, 
and Iterating the method at the forming step. In order to obtain an optimum synchronization plan, It is 
desirable to repeat the entire method described above for the complete set of nodes which are capable 
of being designated as a master clock node. When more than one node is capable of being considered 
as a master clock node, the synchronization planning method is then completed by computing the 
network synchronization perforamnce for each synchronization plan related to a different designated 
master dock node and choosing the synchronization plan which offers the best network synchroni- 
zation performance as computed above. 
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Technical Field 

This invention relates to synchronization of interconnected digital equipment and, more particularly, a 
method of distributing clock signals in a digital communication network. 

5 

Background of the Invention 

Digital communication networks are now appearing in increasingly greater numbers. The digital networks 
are as complex as a switched network providing national telecommunications or as simple as a private data 

10 network having several private branch exchange (PBX) equipments Interconnected by a T1 .5 line. In the digital 
network, a master or reference clock provides continuing timing signals to all other clocks in a digital network. 
This permits all digital equipment on the network to operate from a common time base. Synchronization is made 
possible by designating one or more master clocks as the common time base and distributing their timing infor- 
mation to all equipment in the network. As a result, ail interconnected network equipment are expected to oper- 

15 ate at a rate related to the rate of the designated master clock. 

Proper synchronization of digital equipment on the network would be nearly impossible without such clock 
signals and network performance would be seriously degraded at best A poorly synchronized network would 
experience severe synchronization slips and error bursts, both of which adversely affect throughput, quality, 
and delay performance for data and other traffic and services earned by the digital network. 

20 When two digital communication equipments are transmitting information back and forth to each other at 
different rates, a slower receiving equipment will not be able to keep pace with the faster transmitting equipment 
As a result, the receiving equipment must drop or discard some of the information in order to keep up with the 
current information being received. Depending on equipment characteristics, the speed of one of the equip- 
ments will be adjusted or synchronized to the speed of the other so that both equipments appear to be operating 

25 from the same clock signal. It should be noted that a speed difference causing a timing loss as small as one 
second every 300 years is intolerable according to some telecommunication network standards. 

Typical problems resulting from timing or synchronization errors can vary from exchange of incorrect or 
incomplete information to complete blockage and communication failure of the network. Synchronization errors 
can also propagate downstream through cascaded equipments as a result of a transient timing phase error. 

30 For encrypted data systems, loss or slippage of synchronization can adversely affect security of the network 
by requiring key retransmission. For facsimile transmission on the digital network, synchronization errors can 
lead to blurred or even illegible images causing the user to generate a need for facsimile retransmission. In 
digital video services, synchronization errors can cause picture segments to be distorted and blanked for 
periods extending up to several seconds. 

35 When occurring in industry-specific digital networks, the effect of synchronization errors is further mag- 
nified. Banking networks may lose monetary transaction Information; aviation control networks may be Impaired 
in dealing with the volumes of instantaneous flight information; military and national security can be comprom- 
ised; hotel, automobile, and airline reservation information may be lost; securities exchange information may 
include erroneous data; and internal business communications may become flawed, if occurring at ail. 

40 To address some of these problem areas, various organizations have attempted to set standards concern- 
ing synchronization and clock precision for networks interconnecting digital equipment See, for example, 
American National Standard for Telecommunications — Synchronization Interface Standards for Digital Net- 
works, ANSI T1 .101-1987. While these standards state the error performance of different dock signals, there 
is no attempt to teach or suggest the manner in which one should develop and optimize a synchronization plan 

45 for an entire network to ensure end-to-end network reliability within the acceptance criteria of the standards. 

Summary of the Invention 

Optimized synchronization planning and clock distribution for a network of interconnected digital equipment 
50 is achieved by designating a network node at the highest stratum level as the master clock node, forming a 
group of all unassigned nodes connected to the assign ed node or nodes, selecting subgroup of alt nodes from 
the group wherein the subgroup Includes all nodes having the highest stratum level of the group, limiting the 
subgroup to the nodes which have a desired characteristic when such nodes are Included In the subgroup, 
determining the synchronization performance of each node in the subgroup according to a predetermined criter- 
55 ion, assigning one node from the subgroup as a clock timing receiver wherein the one node exhibits the best 
performance for nodes in the subgroup, end iterating the method at the forming step. 

In order to obtain an optimum synchronization plan, it is desirable to repeat the entire method described 
above for the complete set of nodes which are capable of being designated as a master dock node. When more 
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than one node is capable of being considered as a master clock node, the synchronization planning method 
Is then completed by computing the network synchronization performance for each synchronization plan related 
to a different designated master clock node and choosing the synchronization plan which offers the best network 
synchronization performance as computed above. 

5 

Brief Description of the Drawing 

A more complete understanding of the invention may be obtained by reading the following description of 
a specific illustrative embodiment of the Invention In conjunction with the appended drawing in which: 
10 FIG. 1 is a simplified diagram of a network of interconnected digital equipment; 

FIG. 2 shows a flowchart representation of a method for developing an optimized synchronization plan In 
accordance with the principles of the Invention; and 

FIGs. 3 and 4 show synchronization plans for the network in FIG. 1 developed in accordance with the prin- 
ciples of the invention with master clocks assigned as nodes A and B, respectively. 

15 

Detailed Description 

Digital network are represented as a combination of digital equipment shown as nodes in FIG. 1 intercon- 
nected by links shown as lines In FIG. 1. Synchronization performance of a digital network depends on both 

20 link and equipment performance. Links are required to transmit timing (clock) information accurately throughout 
the digital network. Digital equipment is required to receive the timing (clock) information accurately. In order 
to ensure that ail digital equipment in the network are properly synchronized, preparation of a synchronization 
plan is almost a necessity. 

A synchronization plan describes the flow of timing (clock) information throughout a digital network from 

25 one or more master timing sources (master clock nodes) to the remaining equipment in the network. As the 
size of a digital network grows, preparation of a synchronization plan becomes Increasingly more difficult 
because all possible master timing sources and all possible information paths emanating from the master timing 
sources within the network must be considered. Suboptimal synchronization plans, that is, plans for which 
synchronization performance Is less than optimal such as those developed by prior art adhoc planning methods, 

so are Insufficient to ensure maintenance of network synchronization in accordance with the appropriate standard. 
In digital telecommunication networks, optimum synchronization plans, such as those realized in accordance 
with the principles of the present invention, can permit the digital network to achieve synchronization perform- 
ances which exceed the strict timing requirements required by the standards. 

In a network of synchronous digital equipment, the purpose of an optimum synchronization plan is to trans- 

35 mit timing information from a master timing source to all equipment In the network in the best possible way. 
For such a plan, it is necessary to choose one or more master timing sources as a common time base of the 
network. When properly planned, the network will exhibit a minimum dally time error for all equipment 

In accordance with the principles of this invention, synchronization performance is determined for the net- 
work on the basis of a model which incorporates equipment and link performance characteristics. On the basts 

40 of this model, it is possible to generate an optimum synchronization plan for the digital network. 

In our model, each link in the network has associated therewith a performance parameter specifying the 
average number of daily transmission errors (dte) measured over that link. Daily transmission errors affect the 
accurate transfer of timing information from one equipment to another. The link performance parameter is typi- 
cally a function of the link length and the media type, for example, fiber, satellite, radio, wire and the like. 

45 Every equipment has three parameters representing timing performance of the particular equipment within 
a network. These parameters are the stratum level of the equipment, its timing reaction to errors in the timing 
information sent to it, and errors produced by the clock within the particular equipment 

The stratum level refers to four defined levels of clock performance (1 through 4) as defined in standards 
documents such as the ANSI standard cited above. Smaller stratum level numbers represent more accurate 

so timing performance. Particularly, a stratum 1 clock occupies the highest level in the synchronization hierarchy 
and is known as the frequency reference (common time base) for the entire synchronization network. It is a 
primary frequency standard with a minimum accuracy defined to be better than 1x10-". A stratum 2 dock 
occupies the second level In the synchronization hierarchy. This clock must have or must be adjustable to mini- 
mum accuracy of 1.6x10*. A stratum 3 clock occupies the third level in the synchronization hierarchy and has 

55 or must be adjustable to have a minimum accuracy of 4.6x1 0 4 . A stratum 4 clock occupies the fourth level in 
the synchronization hierarchy and has or must be adjustable to have a minimum accuracy of 3.2x1 0-*. Typically, 
clocks at stratum levels 1, 2 and 3 utilize phase buildout routines to handle clock Interruptions and control resi- 
dual time error. 
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The second equipment performance parameter Is called time interval error (TIE). TIE Is based on the vari- 
ation of the time delay of a given timing (clock) signal with respect to an Ideal timing signal. TIE over a particular 
period Is defined as the difference between the time delay values measured at the end and the beginning of 
the particular period. TIE is generally expressed in microseconds. This parameter characterizes the ability of 
5 equipment to accurately receive timing (clock) information over a link which may be subject to errors. 

The third equipment performance parameter is called propagation (prop). Upon receipt of an error In the 
timing information, some equipment produce errors on its own outgoing communication links. These errors 
affect synchronization performance of other equipment which receive timing Information on the outgoing com- 
munication links. When a particular equipment incorporates a phase buiidout routine which can mask incoming 
10 errors, the equipment does not produce or propagate errors. Propagation effect of a particular equipment Is 
measured as the number of errors (typically ranging from 0 to 2) produced for each incoming error seen by the 
particular equipment 

A network of interconnected synchronous digital equipment can be represented by a graph of nodes and 
links like the one depicted in FIG. 1. Nodes A through G designated as elements 10 through 16 In the graph 
is represent the synchronous digital equipment in the network. Links 20 through 28 In the graph represent the 
interconnection of these equipment or nodes. In this network representation, the digital interconnections be- 
tween the equipment are any medium of communications such as wire, fiber, satellite, radio and the like. 

Each node and link in FIG. 1 are defined by their respective performance parameters. 
These parameters are shown in the following tables below: 

20 

NODE STRATUM PROPAGATION TIE BUILDOUT 
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LINK DTE 
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10 


21 (AD) 
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22 (AE) 
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23 (CD) 
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24 (DE) 
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25 (CT) 
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26 (DG) 
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27 (FG) 


4 


28 (EG) 


12 



55 

With the network represented as shown In FIG. 1, Including the performance parameters of nodes and links, 
the inventive method determines the master timing source for the digital network and develops an optimum 
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synchronization plan. The master timing source is chosen to operate on its own time base. As the inventive 
method is applied, flow of timing information from the master timing source to other equipment is represented 
by arrows on the links as shown In FIGs. 3 and 4. Arrows are shown pointing from the timing source equipment 
to the riming receiving equipment 

5 A more detailed explanation of our inventive method for synchronizing interconnected digital equipment 
proceeds as shown In FIG. 2. 

In step 1, instruction box 200 calls for designation of a master clock node. If there are stratum level 1 nodes 
in the network, then assign all stratum 1 nodes as master nodes. Master nodes which generate the common 
time base do not have a timing assignment, if there are no stratum level 1 nodes, then select the node with the 

10 best stratum level between levels 2 through 4. If there is more than one node having the highest stratum level 
between levels 2 through 4, then choose one of them arbitrarily and call it the master node. The master node 
is now said to be assigned. Daily time error and daily output error, that is, number of errors produced by the 
node, of the master node are considered zero. After instruction box 200, control is passed to step 2 at instruction 
box 202 via line 201. 

is In step 2, instruction box 202 calls for forming a group of all unassigned nodes connected to the assigned 
node, selecting a subgroup of all nodes from the group wherein the subgroup includes all nodes having the 
highest stratum level of the group and limiting the subgroup to nodes which have a common desired charac- 
teristic. This step requires one to select a subset of nodes which are connected to a master node and select 
the best stratum level among them. If more than one node has the best stratum level, select ail of them. If any 
20 of the nodes In the chosen subset produce synchronization impairments which affect downstream clocks (i.e., 
the particular equipment does not incorporate phase buildout), these nodes are to be removed from the chosen 
subset If all the nodes in the chosen subset produce impairments, the chosen subset is left intact Control is 
transferred from instruction box 202 to instruction box 204 via line 203. 

In step 3, instruction box 204 calls for determining the synchronization performance of each node in the 
25 subgroup according to a predetermined performance criterion and assigning one node from the subgroup as 
a clock timing receiver wherein the newly assigned node exhibits the best performance for nodes in the sub- 
group. At the previous step, a subset of nodes was chosen as possible receiving nodes. Based on whether the 
nodes produce Impairments, that is, whether they do not have phase buildout, the following condition is defined 
for each node within the chosen subset CONDJ =TRUE, if a receiving node within the chosen subset has 
30 phase buildout; COND_1=FALSE, otherwise. 

Similarly, for each assigned node connected to a node within the chosen subset (I.e., the possible sources 
of timing), the following condition is defined: COND_2=TRUE, if the possible source of timing has phase build- 
out; COND_2=FALSE, otherwise. 

For the chosen nodes and the respective possible sources of timing for which CONDJ and COND_2 are 
35 true, daily time error of the chosen nodes is calculated as follows: 
daily time_error(receiver)=[dte+daily_ouput_error(source)] 
x TlE(receiver) 
+ daily_time_error(source). 
For the chosen nodes for which CONDJ is true and the respective possible sources of timing with COND_2 
40 Is false, daily time error of the chosen nodes is calculated as follows: 
dailyJime_error(receiver)=[dte + daiiyjouput_error(source)] 
x TiE(receiver). 

After performing these computations, it Is possible to choose the node with the lowest daily time error over 
the links from an assigned node. This node is assigned as a timing receiver. It will receive timing from an assig- 
ns ned node, such as the master, over the link between them which results in the lowest daily timing error. Since 
the chosen node has now been assigned timing, an arrow is placed on the link pointing from the source of timing 
to the chosen node for which timing has been assigned. Control is passed from instruction box 204 to decision 
box 206 via line 205. 

If CONDJ is false for the chosen nodes, then calculate the daily ouput errors produced by the chosen 
60 node as follows: 

daJly_oupuLe*Tor(receiver)=|dte + daily_ouput_error(source)] 
x prop(recelver). 

After this computation, it Is possible to choose the node with the lowest daily ouput error over the links from an 
assigned node and assign that node as a timing receiver. Based on the respective possible timing sources for 
55 the node assigned as the timing receiver, one can calculate the daily time error as in the first equation, if 
COND_2 is true, or in the second equation, if COND_2 is false. 

The node assigned as timing receiver will receive timing from an assigned node, such as the master, over 
the link between them which results in the lowest daily timing error. An arrow is placed on the link pointing from 
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the source of timing to the timing assigned chosen node. Control is transferred from instruction box 204 to deci- 
sion box 206 via line 205. 

In step 4 t decision box 206 calls for determining whether all nodes have been assigned as timing receivers. 
If all nodes have been assigned then control Is transferred to decision box 21 1 via line 207. If all nodes have 

5 not been assigned, then control Is transferred to Instruction box 209 via line 208. If there are nodes which have 
not been assigned, then it Is necessary to select the subset of nodes connected to nodes which already have 
assigned timing sources and select the best stratum level among them. This step is similar to step 2 performed 
at Instruction box 202. If more than one node has the best stratum level, ail are selected. If any of the nodes 
In the chosen subset produces impairments which effect downstream clocks (i.e., do not incorporate phase 

10 buildout), these nodes are removed from the chosen subset If all the nodes in the chosen subset produce 
impairments, do not remove any nodes from the chosen subset Control Is transferred as described above. 

In decision box 211, ft Is necessary to determine whether there are more nodes In the network with the 
samestratum level as the newly assigned master node. If such additional nodes exist, then control Is transferred 
to instruction box 200 via line 213. If no such additional nodes exist, then control is transferred to instruction 

is 214 via line 212. When control la transferred from decision box 21 1 to Instruction box 200, ft Is necessary to 
select a different node to be assigned as the master node. 

In step 5, instruction 21 4 calls for computing the network synchronization performance for each synchroni- 
zation plan related to a different designated master clock node and, thereafter, choosing the synchronization 
plan which offers the best computed network synchronization performance. After considering ail the network 

20 synchronization plans with the various nodes as possible master, it is necessary to calculate the network 
synchronization performance. This performance is calculated as follows: 



25 netjync jerfe £ f(daily_time_error(node), numbcrof_links(node)) 

nodes 

where £ represents the summation of the synchronization performance over all the nodes in the network. After 
computing the network synchronization performance, it is possible to choose the network synchronization plan 
30 with the best (i.e., the lowest) network synchronization performance. The timing master for the network will be 
the master node Indicated In that synchronization plan. The flow of synchronization timing information through 
the network is Indicated by each timing assignment (arrow direction). 

EXAMPLE 

35 

To illustrate the above-described method, the telecommunications network of FIG. 1 is considered. In this 
example, there are seven digital equipment named A through G that are interconnected as shown in FIG, 1. 
The performance parameters of each equipment and link have been indicated in the tables above. 

The method to find the optimal synchronization plan proceeds through several iterations of the method. 
40 Each Iteration Is described in detail below. The number of iterations will be equal to the number of nodes receiv- 
ing timing. The total number of iterations increases linearly with the number of possible masters. 

Iteration 1.~for Master = Node A 

45 Step 1. The best stratum level of all nodes in the network is 2. There are two nodes with the best stratum levels, 
namely nodes A and B. Initially, select node A as the master node. 

Step 2. Nodes B, D, and E are connected to node A. Among these three nodes, node B has the best stratum 
level. Therefore, node B Is selected as the best node among B, D, and E. 

Step 3. Since there is only one node selected at Step 3, and since there Is only one link between A and B, 
so node B Is assigned timing from node A over link AB. The daily timing error of node B is 3 microseconds and 
its daily output errors is 0. Therefore, nodes A and B are assigned timing. 

Step 4. Nodes D and E are the only unassigned nodes connected to the timing assigned nodes A and B. Since 
both nodes D and E have the same stratum level, they are both selected as the best nodes. Both nodes D and 
E have phase buildout and do not produce Impairments, so they remain in the chosen subset 

55 

iteration 2$or Master « Node A 

Step 3. Calculate the daily time error performance of node D over link AD and of node E over link AE as follows: 
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Node 0 daily_tlme_error = (5 + 0)X 0.07 * 0.35 
Node E daily_tfme_errorj=_(20 + 0)X 0.65 = 13.0 

Since the performance of node D is better than node E, node D is assigned the timing source node A over 
link AD. The daOy number of output errors from node D is zero. Therefore, nodes A.B, and D are assigned timing. 
5 Step 4. Nodes C, E, and G are the only unassigned nodes connected to the timing assigned nodes A, B, 
and D. Node E has the best stratum level and is chosen as the best node. 

iteration 3 for Master = Node A 

10 Step 3. Calculate the daily time error performance of node E over links AE and DE as follows: 
Link AE dafly_tIme_error = (20 + 0) x 0.65 - 1 3.0 
Link DE daily_time_enror = (3 + 0)x 0.65 + 0.35 = 2.3 

Since the performance of node E is better when timing comes from node D over link DE than from node A over 
link AE, node E is assigned the timing source node D over link DE. The daily number of output errors from node 
15 E is 0. Therefore, nodes A, B, D, and E are assigned timing. 

Step 4. Nodes C and G are the only unassigned nodes connected to the timing assigned nodes A, B, D, and 
E. They both have the same stratum level and, therefore, are both chosen. Both nodes C and G do not incor- 
porate phase buildout and produce errors, thus they are both chosen. 

20 iteration 4: for Master = Node A 

Step 3. Since both chosen nodes produce impairments, the daily output errors of the nodes are calculated. Cal- 
culate the daily output errors of node C over link CD and node G over links DG and EG as follows: 
Node C, Link CD daiIy_outputerrar = (4 + 0) X 2 = 8 
25 Node G, Link DG dal!y_output_error ■ (7 + 0) X 1 = 7 
Node G, Link EG daily_output_error = (1 2 + 0} X 1 = 12 

Since the number of daily output errors is lower for node G than node C, node G will be assigned timing. Cal- 
culate the daily time error of node G over links DG and EG as follows: 
Link DG dailyjime_error = (7 + 0) X 1 00 + 0.35 « 700.35 
30 Link EG dailyJime_error = (12 + 0) X 100 + 2. 3 = 1202. 3 

Since the performance of node G is better when timing comes from node D over link DG than from node E over 
link EG, node G is assigned the timing source node D over link DG. The daily number of output errors from 
node G is 7. Therefore, nodes A, B, C, E, and G are assigned timing. 

Step 4. Nodes C and F are connected to the assigned nodes D and G. Among these two nodes, node F has 
35 the best stratum level. Therefore, node F is selected as the best node. 

iteration 5: for Master = Node A 

Step 3. Since there Is only one node selected at Step 2, and since there is only one link between G and F, 
40 node F Ss assigned riming from node G over link FG. The daily timing error of node F is given by 

NodeF daily Jime_error = (4 + 7)X1.0 « 11.0 
The daOy number of output errors is 0. Therefore, ail nodes except C are assigned timing. 
Step 4. Since node C Is the only node without a timing assignment, it is chosen. 

45 iteration 6 for Master = Node A 

Step 3. Calculate the daily time error performance of Node C over link CD and CF as follows: 
Link CD daDy_time_error=(4+0)X250+3.35=1 000.35 
Link CF daHyJime_error=<3+0)X250+1 1 .0=761 .0 
so Since the performance of node C is better when timing comes from node F over link CF than from node D over 
link CD, node C Is assigned the timing source node F over link CF. 

Step 4. All nodes are assigned timing. There is one node, node B, with the same stratum level as node A, which 
has not been chosen as a possible master node. Go to step 1 and choose node B as the master node. 

55 iterations for Master = Node B 

There will be a total of six (6) iterations for Master = node B. The results of these iterations is shown In FIG.4. 
Step.5 When the procedure has been repeated with node B as the master node, all nodes with the best stratum 
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level will have been chosen as a possible master node. 

The computation results which are used to develop the synchronization plans for the digital network of FIG.1 
as shown in FIGs.3 and 4 are show. In the following tables. The first table is developed using node A as the 
master.the second table is developed using node B as the master. 
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NODE TIME ERROR OUTPUT ERROR 
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40 Step 5 of the method is used to compare the network synchronization performance of the two possible plans 
(one with node A as the timing master and one with node B as the timing master). The network performance 
for the plan using node A as the timing master is better than the o ne using node B. Thus, the first synchronization 
plan, with A being the timing master, will be the synchronization plan for the network. 



43 

Claims 

1 . A method for planning synchronization dock distribution for a network of interconnected digital equipment 
wherein each digital equipment Is represented as a node interconnected to other nodes by a links, the 

60 method comprising assigning one node at the highest stratum level as the master clock node, forming a 
group of all unassigned nodes connected to the assigned node or nodes, selecting a subgroup of ail nodes 
from the group wherein the subgroup Includes all nodes having the highest stratum level of the group, limi- 
ting the subgroup to nodes which have a desired characteristic when such nodes are included in the sub- 
group, determining synchronization performance of each node in the subgroup according to a 

55 predetermined criterion, assigning one node from the subgroup as a clock timing receiver wherein the one 
node exhibits the best performance for nodes in the subgroup, and iterating the method at the forming step 
until all nodes are assigned as timing receivers. 
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Z The method as defined in claim 1 further comprising assigning a different node having the highest stratum 
level as the master clock node and performing the iteraring step. 

3. The method as defined in claim 2 further comprising computing network synchronization performance for 
5 each synchronization plan related to a different assigned master clock node and choosing the synchroni- 
zation plan which offers an optimum network synchronization performance. 

4. The method as defined In claim 1 comprising indicating a timing flow along a link into the clock timing 
receiver after the receiver has been assigned. 

10 

5. The method as defined in claim 1 wherein the desired characteristic includes the existence of phase build- 
out for the equipment associated with the node. 
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FIG. 2 
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